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- - - Abstract 

. This paper presents a unified analysis framework that captures recent advances in the 

P^ I study of local-optimaUty characterizations for codes on graphs. These local-optimality 

^f.' characterizations are based on combinatorial structures embedded in the Tanner graph of 

^ . the code. Local-optimality implies both unique maximum-likelihood (ML) optimality and 

■<^ I unique linear-programming (LP) decoding optimality. Also, an iterative message-passing 

_j. ' decoding algorithm is guaranteed to find the unique locally-optimal codeword, if one ex- 

ists. 

We demonstrate this proof technique by considering a definition of local-optimality 
that is based on the simplest combinatorial structures in Tanner graphs, namely, paths of 
length h. We apply the technique of local-optimality to a family of Tanner codes. Inverse 
polynomial bounds in the code length are proved on the word error probability of LP- 
decoding for this family of Tanner codes. 
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^ : 1 Introduction 

00 . 

The method of decoding error correcting codes based on a linear programming (LP) relax- 
ed ! ation of maximum-likelihood (ML) decoding was introduced by Feldman, Wainwright and 
^ I Karger IIFK021 IFWKOSn . Successful decoding by LP-decoding was analyzed for different 

codes and channels (see e.g., llFMS+07[lFS05llDDKW08LISkallll \ Significant advances in the 



X 



analysis of successful LP-decoding have been achieved recently. Following Koetter and Von- 
tobel [IKV06II . Arora et al. [ ADS09II obtained improved bounds for (3, 6)-regular low-density 



^ . parity-check (LDPC) codes over the binary symmetric channel (BSC). These techniques were 

■ - -' extended to memoryless binary-input output- symmetric (MBIOS) channels HHEllH and to Tan- 

ner codes [lEHlllI . The proofs in these papers are based on complicated graphical structures and 
on a sophisticated analysis of a random min-sum process. Our goal in this paper is to present a 
simple analysis based on this proof technique that proves nontrivial bounds for a broad family 
of Tanner codes. 
The proof technique in [IADS09II is based on the following steps: 
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1. Define a set of deviations. A deviation is induced by combinatorial structures in the 
Tanner graph or the computation tree [IWib96l IKV061 IADS09I IVonlOi IEHTH . 

2. Define local-optimality. Local-optimality is a combinatorial characterization of a code- 
word X G {0, 1}^ with respect to a channel output A G R.^. This definition is based on: 

(1) a definition of a relative point for a codeword x and each deviation /3 HFelOBH . and 

(2) a definition of the cost of each relative point with respect to the log-likelihood ratios 
(LLR) vector A. Loosely speaking, a codeword x is locally-optimal if its cost is smaller 
than the cost of every relative point. 

3. Prove that if x is locally-optimal codeword w.r.t. an LLR vector A, then x is the unique 
maximum-likelihood (ML) codeword. The proof is based on a decomposition lemma 
that states that every codeword is a conical sum of deviations. 

4. Prove that if x is locally-optimal codeword w.r.t. an LLR vector A, then x is the unique 
linear-programming (LP) codeword. This proof is based on a lifting lemma | HE11 1 that 
states that local-optimality is invariant under liftings of codewords to covering graphs. 

5. Analyze the probability that there does not exist a locally-optimal codeword. 

We demonstrate this proof technique by considering the simplest combinatorial structures 
in Tanner graphs, namely, paths of length h. We attach to each path p a deviation (3 G R^ 
that equals the multiplicity of each variable node along p divided by its degree times h + 1. 
Surprisingly, all the ingredients of the proof technique can be demonstrated with respect to this 
primitive set of deviations. 

The family of codes we consider is the set of Tanner codes that satisfy two properties: (1) the 
local codes contain only codewords of even weight, and (2) all the variable node degrees are 
even. We refer to this family of Tanner codes as even Tanner codes^ Among the codes in 
this family are: (1) LDPC codes with even left degrees, (2) irregular repeat accumulate codes 
where the repetition factors are even, and (3) expander codes with even variable node degrees 
and even weighted local codes. 

Apart from offering a simple application of a powerful proof technique, we shed light on 
two issues. (1) The deviations should not be limited by the girth of the Tanner graph. In- 
deed, we consider paths of arbitrary length h which need not be simple for steps 1-4 of the 
proof technique. One should note that step 5 that bounds the probability of the existence of 
a locally-optimal codeword requires independence of the LLRs in the deviation. Hence, the 
bound here holds only for simple paths, thus limiting h by the girth. (2) The Tanner graph need 
not be regular. Irregular degrees are handled by dividing the deviations by the node degrees 
(see [Von lOl). 

Local-optimality is also related to iterative decoding. An iterative message-passing decod- 
ing algorithm is presented in HEHllI with the guarantee that, if a locally-optimal codeword 
exists, then the decoding algorithm finds it. Moreover, since local-optimality can be verified 
efficiently, an ML-certificate is obtained whenever there exists a locally-optimal codeword. 

We prove inverse polynomial bounds in the code length on the word error probability for 
LP-decoding of even Tanner codes whose Tanner graphs have logarithmic girth. For certain 



' The parity of the node degrees is used to prove that the subgraph induced by codewords is Eulerian, a key 
component in the decomposition lemma of step 3. 



sub-classes of even Tanner codes this technique provides new bounds on the word error prob- 
ability of LP-decoding. Li the case of repeat accumulate codes, we provide a simple proof of 
previous results [1FK02[IGBTT]| . 

Organization. The remainder of the paper is organized as follows. Section |2]provides back- 
ground on ML-decoding and LP-decoding of Tanner codes over MBIOS channels. Section [3] 
contains steps 1-4 in the proof technique of local-optimality applied to even Tanner codes. 
In Section |4] we demonstrate step 5 by a simple probabilistic analysis of the event where no 
locally-optimal codeword exists. We conclude with a discussion in SectionUl 

2 Preliminaries 

Graph Terminology. Let G = (V, E) denote an undirected graph. Let Ng{v) denote the set 
of neighbors of node v &V . Let degg.(t>) = |A/G'(f )| denote the edge degree of node v in graph 
G. A path p = (t>, . . . , m) in G is a sequence of vertices such that there exists an edge between 
every two consecutive nodes in the sequence p. A simple path is a path with no repeated vertex. 
A simple cycle is a closed path where the only repeated vertex is the first and last vertex. A 
path p is backtrackless if every three consecutive vertices along p are distinct (i.e., a subpath 
(m, f , u) is not allowed). Let \p\ denote the length of a path p, i.e., the number of edges in p. Let 
girth(G') denote the length of the shortest cycle in G. The subgraph of G induced by S (^V 
consists of S and all edges in E, both endpoints of which are contained in S. Let Gs denote 
the subgraph of G induced by S. 

Tanner-codes and Tanner graph representation. Let G = {V U J,E) denote an edge- 
labeled bipartite-graph, where V = {f i, . . . , v^} is a set of N vertices called variable nodes, 
and J = {Ci, . . . , Gj} is a set of J vertices called local-code nodes. We associate with each 

local-code node Gj a linear code C of length deg(j(Cj). Let C ={C :l^j^j} denote 

the set of local codes, one for each local-code node. We say that Vi participates in C if (f j, Gj) 
is an edge in E. 

A word X = (xi, . . . , xn) G {0, 1}^ is an assignment to variable nodes in V where Xj is 
assigned to Vi. Let Vj denote the set Mg{Gj) ordered according to labels of edges incident to 
Gj. Denote by xy^ G {0, \\'^'^^g{Cj) tj^g projection of the word x = (xi, . . . , xn) onto entries 
associated with Vj. 

The Tanner code C{G, C ) based on the labeled Tanner graph G is the set of vectors x G 
{0, 1}^ such that xy^ is a codeword in C for every j e {1, . . . , J}. Let C^ denote the extension 
of the local code C from length deg(Cj) to length N defined by C^ = {x G {0, 1}^ | xy^ G 
C }. The Tanner code is simply the intersection of the extensions of the local codes, i.e.. 

We consider a family of Tanner codes defined as follows. 

Definition 1 (even Tanner codes). A Tanner code C{G,C ) based on a Tanner graph G = 

(V U J', E) is called an even Tanner code if: (1) degQ{v) is even for every v eV, and (2) every 

—j — J 
codeword in each local code C G C has even weight. 



LP decoding of Tanner codes over memoryless channels. Let Cj G {0, 1} denote the 
zth transmitted binary symbol (channel input), and let yi E H denote the ith received sym- 
bol (channel output). A memoryless binary-input output-symmetric (MBIOS) channel is de- 
fined by a conditional probability density function f{yi\ci = a) for a E {0, 1}, that satisfies 
f{yi\0) = f{—yi\l). The binary erasure channel (BBC), binary symmetric channel (BSC) 
and binary-input additive white Gaussian noise (BLAWGN) channel are examples for MBIOS 
channels. In MBIOS channels, the log-likelihood ratio (LLR) vector A E R^ is defined by 
KiVi) — In ( {( ' e'=i) ) ^°^ every input bit i. For a code C, Maximum-Likelihood (ML) decoding 
is equivalent to 

x'^^^y) = aigmm{X{y),x), (1) 

a;£conv(C) 

where conv(C) C [0, 1]^ denotes the convex hull of the codewords in C. 

In general, solving the optimization problem in ([T) for linear codes is intractable [|BMvT781 . 
Feldman et al. [|Fel03[ IFWKOSn introduced a linear programming relaxation for the problem of 
ML decoding of Tanner codes with single parity-check codes acting as local codes. We consider 
an extension of this definition to the case in which the local codes are arbitrary as follows. The 

generalized fundamental polytope V = V{G, C ) of a Tanner code C = C{G, C ) is defined 
by 

P= Pi conv(C-'). (2) 

C:ieCJ' 

Given an LLR vector A for a received word y, LP-decoding is defined by the following 
linear program: 

x^^{y) = argmin (A(?/),x). (3) 

The difference between ML-decoding and LP-decoding is that the fundamental polytope V{G,C ] 
may strictly contain the convex hull of C. Vertices of V{G, C ) that are not codewords of C 
must have fractional components and are called pseudocodewords. 

3 A Local Combinatorial Certificate for an Optimal Code- 
word 

In this section we define a simple type of local-optimality characterization that is based on 
backtrackless paths of arbitrary length h in the Tanner graph. We prove that for codewords of 
even Tanner codes, this characterization suffices both for ML-optimality and LP-optimality. 

Definition 2 (normalized characteristic vector). Consider a Tanner graph G = (V U J', E). 
The normalized characteristic vector Xg{p) G R'^' of a path p is defined as follows. For every 
V E V the component [xg(p)]i; equals to the multiplicity of the variable node v in the path p 
divided by its degree in G. Formally, 

Ifp is closed (i.e., a cycle), then we count the multiplicity of the endpoints only once. 



The normalization by degQ{v) is needed if G has irregular variable node degrees. 

For any fixed h, let B^'^^ denote the set of normalized characteristic vectors of backtrackless 
paths of length h'mG scaled by a factor j^. That is, 
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< — p is a backtrackless path of length h>. (5) 



Vectors in B^'^^ are called deviations. Note that B^'^^ E [0, l]'^' because every variable node 
appears less than h + 1 times in a path of length h. 

For two vectors x G {0, 1}^ and / G [0, 1]^, let x © / G [0, 1]^ denote the relative point 
defined by {x © /)j = \xi — fi\ IIFel03ll . The following definition characterizes local-optimality 
based on backtrackless paths for even Tanner codes over MBIOS channels. 

Definition 3 (path-based local-optimality). Let C{G) C {0, 1}^ denote an even Tanner code 
and let h G 1N+. A codeword x G C{G) is /i-locally optimal with respect to A G R^ if for all 
vectors f3 G B^^\ 

(A,x©/3) > (A,x). (6) 

For two vectors y, z E R^, let "*" denote coordinate- wise multiplication, i.e., {y*z)i = yi- 
Zi. For a word x G {0, 1}^, let (—1)^ G {±1}^ denote the vector whose ith component equals 
(—1)^'. The following proposition and corollary state that the mapping (x, A) i— )■ (0^, (— 1)"^ * 
A) preserves local-optimality. 

Proposition 4. jJEEHil For every A G R^ and every /3 G [0, 1]^, 

((-l)^*A,/3) = (A,x©/3)-(A,a;). (7) 

Corollary 5 (symmetry of local-optimality). For every x G C, x is h-locally optimal w.r.t. A if 
and only ifO^ is h-locally optimal w.r.t. (—1)^ * A. 

Proof By Proposition H (A, x ® (5) - (A, x) = ((-1)^ * A, /3). D 

Corollary |5] suggests that a codeword x can be verified to be /i-locally optimal w.r.t. a given 
LLR A by verifying that each backtrackless path of length h has positive normalized cost w.r.t. 
(—1)^' * A. That is, if the minimum normalized cost of every path with length h w.r.t. (—1)^ * A 
is positive, then x is /i-locally optimal w.r.t. A. A min-cost path of length h in a graph can be 
computed by a simple dynamic programming algorithm (Floyd's algorithm) in time 0{h ■]£]). 
Hence, a codeword can be efficiently verified to be locally-optimal w.r.t. A. 

3.1 Local-Optimality Implies ML-Optimality 

In the following section we show that local-optimality is sufficient for ML-optimality (Theo- 
rem[8]). The proof of Theorem |8]is based on the representation of every codeword as a conical 
combination of deviations in i3*^^^ (Corollary |7]). We first prove that every codeword in an even 
Tanner code is a conical combination of normalized characteristic vectors of simple cycles in 
the Tanner graph (Lemma |6]). Then we show that every cycle of length £ is a conical combina- 
tion of i deviations in B^^^ for any arbitrary h, which implies Corollary |7] 



Lemma 6 (simple cycles decomposition). Let C{G) denote an even Tanner code, and let V 
denote the set of simple cycles in G. For every codeword x ^ 0^, there exists a distribution p 
over the set T and an a > 1, such that 



X 



a-E^epr[xG(7)]- (8) 



Proof. Let Vx — {v \ x^ = 1}, and let G^. denote the subgraph of the Tanner graph G induced 
by Vx U Mg(Vx)- Because x is a codeword in an even Tanner code, the degree of every node 
(both variable nodes and local-code nodes) in Gx is even. Therefore, each connected compo- 
nent in Gx is Eulerian. Denote by {Gj } the set of connected components of Gx, and let ip^^'' 
denote an Eulerian cycle in Gj . 

Consider a variable node v in the connected component Gj , then the multiplicity of v in 
ip^^'^ equals -£i^M_ Therefore, 2 • ^j[XG{'ip''^'')]v = 1 (and, by definition, Xi, = 1). 

Every Eulerian cycle ^^^^ can be decomposed into a set of edge disjoint simple cycles. Let 
r'^-') denote the decomposition of ijj'^^'^ into simple cycles. Then, Xg(V'*^''^) = Z^^ero) Xcil)- 
Thus, 

x^ = 2-Y^ Y^ Xg{i)- 
j 7erU) 

Let p denote the uniform distribution over U^T*^-'^ and let s = | U^T'^-'^ | . Then, 

D 

Corollary 7. Let C{G) denote an even Tanner code, and let h G 1N+. For every codeword 
X 7^ 0^, there exists a distribution p over the set B^^^ and an a' > I such that 

Proof. Following Lemma [6l it suffices to show that, for every simple cycle 7, the set {'ipi} of 
paths in 7 of length h satisfy 



35, > 1 : Xcil) = ^7 ■ E hTl ■ ^^^"^^^^ ^^^ 

1=0 

Indeed, let f3i{-f) = ^ ■ Xg{A), then 

x = a-Yp{-f) -XGil) 

7 

Note that in the last line, we use the fact that the cycles in Gx are decomposed into edge disjoint 
cycles, and hence each deviation (3 appears in exactly one cycle, denoted by 7^. The corollary 
follows because the coefficients p(7/3) ■ 6^ are nonnegative and their sum is at least one. 



We now prove Equation ^. Let 7 = (fo, f 1, . . . , f£_i, f^ = fo) be a simple cycle in G of 
length i. For every ^ i ^ i — 1, lei ^pi = (f j, Vi+i mod £, •••, Vi+h mod £) denote a segment of 
7 that starts at node Vi and contains h edges. (Note that if h ^ girth (G), then a single segment 
may traverse a node in the cycle more than once.) For every ^ j ^ h, a node v appears 
exactly once as the jth node in one of the paths {^iJilQ. 

If h is not a multiple of i, then the multiplicity of every vertex t> G 7 in U^ZoV^j equals h + 1. 
Therefore, 

i=0 

Otherwise, i divides h, and every ipi is a cycle whose both endpoints Vi are counted as one 
occurrence in xd^i)- Hence, the multiplicity of every vertex t> G 7 in U^~qV'j equals h, and 

Xcil) = —^ Y^ -j^^ ■ Xcii^i)- 

i=0 

D 

Theorem 8 (local-optimality is sufficient for ML). Let C{G) denote an even Tanner code. Let 
A G R^ denote the LLR vector received from the channel and let h G 1N+. If x is h-locally 
optimal codeword w.r.t. A, then x is also the unique maximum-likelihood codeword w.r.t. A. 

Proof. The proof follows nADS09[ proof of Theorem 2] and HHElll proof of Theorem 6]. 
We use the decomposition proved in Corollary |7]to show that for every codeword x' 7^ x, 

(A, x') > (A, x). Let z = X ® x'. By linearity, z G C{G). Moreover, 2; 7^ 0^ because x 7^ x'. 
By Corollary |7] there exists a distribution over the set B^'^\ such that E^gg(h)/3 = S ■ z, where 
5 = ^ < 1. Let / : [0, 1]^ ^ R be the affine linear function defined by /(/3) 4 (A, x © /3) = 
(A,a;J+Eti(-irA,A.Then, 

(A, x) < E^gBih) (A, X ® l3) (by local-optimality of x) 

= (A, X © E^gg(h)/3) (by linearity of / and linearity of expectation) 

= (A, X © 6z) (by Lemma[6]) 

= {X,{l-6)x + 6{x®z)) 

= {X, {1 - 6)x + 6x') 

= il-5){X,x) + 6{X,x'). 

which implies that (A, x') > (A, x) as desired. D 

Remark: Lemma [6] allows one to define local-optimality with respect to deviations induced 
by simple cycles. Local-optimality based on backtrackless paths of arbitrary length h decouples 
the definition of local-optimality from the girth of the Tanner graph. The implication of this 
decoupling on iterative decoding is discussed in Section |5l 

3.2 Local-Optimality Implies LP-Optimality 

In the following section we show that local-optimality is sufficient for LP-optimality (Theo- 
rem [TT]). We consider graph cover decoding introduced by Vontobel and Koetter HVKOSil and 



its extension to Tanner codes HHallli Chapter 2.6]. The proof of Theorem [TT]is based on 
Lemma [To] that states that local-optimality is preserved under lifting to any Af -cover graph. 

The presentation in this section uses the terms and notation of Vontobel and Koetter HVKOSn 
(see also IIHallli Chapter 2.6]). Let G denote an M-cover of G. Let x = x^'^' G C{G) and 
A = A'''^ e R^'^ denote the A/-lifts of x and A, respectively. 

Proposition 9 (local-optimality of the all-zero codeword is preserved by M-lifts). 0^ is h- 
locally optimal codeword w.r.t. A G R^ if and only ifO'^'^^ is h-locally optimal codeword w.r.t. 
A. 

Proof. Consider the surjection ip of paths with length /i in G to paths in G. This surjection 
is based on the covering map between G and G. Given a path ^ in G, let if) = v'(V^). Let 

/3 = ^Xg(V') and (5 = /^Xg(V')- The proposition follows because (A, /3) = (A, /3). D 

The following lemma states that local-optimality is preserved by lifting to an M-cover. 

Lemma 10. x is h-locally optimal w.r.t. A if and only ifx is h-locally optimal w.r.t. A. 

Proof. Assume that x is /i-locally optimal codeword w.r.t. A. By Corollary |5J 0^'*^ is h- 
locally optimal w.r.t. (—1)^ * A. By Proposition |9l 0^ is /i-locally optimal w.r.t. (— 1)"^ * A. 
By Corollary m x is /i-locally optimal w.r.t. A. Each of these implications is necessary and 
sufficient, and the lemma follows. D 

The following theorem is obtained as a corollary of Theorem |8] and Lemma [lOl The proof 
is based on arguments utilizing properties of graph cover decoding. Those arguments are used 
for a reduction from ML-optimality to LP-optimality HHElll Theorem 8]. 

Theorem 11 (local-optimality is sufficient for LP optimality). Ifx is a h-locally optimal code- 
word w.r.t. A, then x is also the unique optimal LP solution given A. 

Proof. Suppose that x is /i-locally optimal codeword w.r.t. A G R^. By IIVK051 Proposition 
10], for every basic feasible solution z G [0, 1]^ of the LP, there exists an M-cover G of G and 
an assignment z G {0, 1}^'^^ such that z G C{G) and z = Cl^)^ where C(z) is the image of 
the scaled projection of z in G (i.e., the pseudo-codeword associated with z). Moreover, since 
the number of basic feasible solutions is finite, we conclude that there exists a finite M-cover 
G such that every basic feasible solution of the LP admits a valid assignment in G. 

Let z* denote an optimal LP solution given A. Without loss of generality z* is a basic 
feasible solution. Let z* G {0, 1}^ *^ denote the — 1 assignment in the Af -cover G that 
corresponds to z* G [0, 1]^. By pVKOSi Proposition 10] and the optimality of 2* it follows that 
z* is a codeword in C(G) that minimizes (A, 5) for z G C(G), namely z* is the ML codeword 
inC(G)w.r.t. A^^^. 

Let X = x^*^ denote the Af -lift of an /i-locally optimal codeword x. Note that because x is 
a codeword, i.e., x G {0, 1}^, there is a unique pre-image of x in G, which is the A/-lift of x. 
Lemma [TOl implies that x is /i-locally optimal codeword w.r.t. A. By Theorem [8l we also get 
that X is the ML codeword in C(G) w.r.t. A^^^. Moreover, Theorem |8]guarantees the uniqueness 
of an ML optimal solution. Thus, x = z*. Because x = z*, when projected to G, we get that 
X = z* and uniqueness follows, as required. D 



4 Probabilistic Analysis of Path-Based Local-Optimality 

In the previous section, we showed that LP-decoding succeeds if a locally-optimal codeword 
exists w.r.t. the received LLR. In this section we analyze the probability that a locally-optimal 
codeword exists for even Tanner codes in MBIOS channels. The following equation justifies 
the all-zero codeword assumption for analyses based on local-optimality characterizations. 

Pr{LP decoding fails} = Pt{x ^ x^^{X) \ c = x} 

(1) , I , 

^ Prjx is not /i— locally optimal w.r.t. A c = x| 

= PrjO^ is not /i— locally optimal w.r.t. (—1)^ * Ale = x} 

= PrjO is not /i— locally optimal w.r.t. A|c = } 

= Pr{3/3 G B'^''^ such that (A, /3) ^ 0|c = 0^}. (10) 

Inequality (1) is the contrapositive statement of Theorem [TTl Equality (2) follows Corollary |5] 
For MBIOS channels, Pr(Aj | q = 0) = Pr(— Aj | Cj = 1). Therefore, the mapping (x, A) i— )■ 
(0^,6 * A) where bi = (—1)^' is a measure preserving mapping. Equality (3) follows by 
applying this mapping to (x, 6 * A) t-)- (0^, b* b* \). Equality (4) follows by the definition of 
path-based local-optimality. 

Following Equation (flOl) . our goal is to prove an upper bound on the probability that there 
exists a path of length hinG whose normalized characteristic vector has non-positive cost w.r.t. 
A. 

We use the following notation. For a path ip, the normalized cost ofip w.r.t. A is defined by 
cost{ip) = (A,xg(^))- Let df"" = mm{degGiv) \ v e V}, d^"" = Tua^idegGiv) \v eV}, 
and rf^'^^ = max{de^G(C) \ C e J}.l.e.i D = d^^ ■ dg*^^. 

The intuition of the analysis is that long simple paths are unlikely to have non-positive cost. 
We restrict the path length hy h < girth(G) only for the probabilistic analysis. Tanner graphs 
with logarithmic girth can be constructed explicitly (see e.g. [|Gal63ll ). In particular, we assume 
that girth(G') > log£,(iV). 

The following theorem presents an analytical bound on the word error probability of the LP 
decoder over the BSC. 

Theorem 12. Let C{G) denote an even Tanner code of length N such that g = logj^{N) < 

imin 

girth^G). ConsideraBSCwith crossover probabilityp. Forany e > 0, ifp < D '^l^ 220^ 

then the LP decoder fails to decode the transmitted codeword with a probability of at most N'"^. 

For example, for left-regular codes (i.e., d^^"^ = rf™"^)' set e = |. If p ^ -^^, then the word 
error probability of LP-decoding is at most -4^. 

Proof. By Equation (flOl) we may assume that the all-zero codeword is transmitted, i.e., c = 0^. 
Hence A„ = 1 w.p. (1 — p) and A^, = —1 w.p. p. We bound the word error probability P^ using 
a union bound over all the events where simple paths of length g have non-positive normalized 
cost in G w.r.t. A. 

Let -iphe a particular path of length g; -ip contains | variable nodes. Each variable node in 
the path is assigned +1 with probability (1 — p) and —1 with probability p. Because of the 



jmin 

degree normalization, at least | ■ ^minjr^max of the variable nodes in tp must be assigned —1 to 

. jniin 

obtain cost(V^) ^ 0. Let 6 = ^min^^max • Therefore, 

g 

There are at most | V| ■ D^ different simple paths of length g in G. By the union bound. 



Pr{cost(7^) ^ 0} ^ ( / )p2-'^ ^ 22 ■ pf •^. (11) 



Pw < |V| -Di -2^ -pi-^ 

^ (12) 

D 

For the case of BI-AWGN channel, we derive a bound on the word error probability for 
left-regular even Tanner codes, i.e., di = dY" = df^"^. The extension to the case of irregular 
even Tanner codes requires exhaustive notation and computations. 

Theorem 13. Let C{G) denote a left-regular even Tanner code of length N such that g = 
logjj{N) where D = d^- d^^^. Consider a BI-AWGN channel with variance a"^. For any e > 0, 
if (7^ < "^f^^ , then the LP decoder fails to decode the transmitted codeword with a probability 
of at most . " ■ N-\ 



Proof By Equation (flOl) we assume that the all-zero codeword is transmitted, i.e., c = 0^. 
Hence A^, = 1 + (pi where (pi ~ A/'(0, a^) is a zero-mean Gaussian random variable with 
variance cr^. We bound the word error probability P^j using a union bound over all the events 
where simple paths of length g have non-positive cost in G. 

Let ^ be a particular path of length g. If the sum of the costs of the variable nodes in tp is 
non-positive, then cost(^) ^ 0. Hence, 



Pr{cost{tp) ^ 0} = Pr{ j] (1 + (pi) ^ O} 

9 



':' (13) 



The sum of independent Gaussian random variables (RVs) with zero mean is a zero-mean 
Gaussian RV whose variance equals to the sum of the variances of the accumulated variables. 

9 

Let $ = XliLi </*«' ^^^^ ^ ~ -^(0, a'^ ■ |) is a zero-mean Gaussian RV with variance a^ ■ |. 
Moreover, the Gaussian distribution function is symmetric around 0. Therefore, 

Pr{cost{ij) ^ 0} = Pr{<^ ^ --} 
10 



For a Gaussian RV ~ A/'(0, a"^) with zero mean and variance a^, the inequality 

Pr{(j) ^ x} ^ -^^e~^ (15) 

xv27r 

holds for every x > IIFel68l . We conclude that 

Pr{cost(V^) ^ 0} ^ -^—e~^. (16) 

There are at most | V| ■ D^ different simple paths of length g in G. By the union bound, 

P^ < |V| ■ D2 . Pr{cost(V^) ^ 0} 
< N ■ D^ e 3^ 



ng 

3 a ioSD(N) 

^ A^2 . =e ^^^^ 

V7rlogo(iV) (17) 



a 



a 
y/n\ogo{N) 



Ari-3^^°gD(^) 



^ ^ ■ AT-. 
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5 Discussion 

5.1 Finding Locally-Optimal Codewords with Message-Passing Algorithm 

A message-passing decoding algorithm, called normalized weighted min-sum (NWMS), was 
presented in HEHllll for Tanner codes with single parity-check (SPC) local codes. The NWMS 
decoder is guaranteed to compute the ML-codeword in h iterations provided that a locally- 
optimal codeword with height h exists. The local-optimality characterization in [ EH 11,1 is 
stronger and is based on subtrees of computation trees of the Tanner graph. We note that 
for even Tanner codes with SPC local codes, the NWMS decoder with uniform weights (i.e., 
w = 1^) is guaranteed to compute the ML-codeword in h iterations provided that a path-based 
/i-locally optimal codeword exists. The number of iterations h may exceed the girth of the 
Tanner graph. 

Consider an even Tanner code and an LLR vector A. Assume that there exists a codeword 
X that is /i-locally optimal w.r.t. A. Given this assumption, maximum-likelihood decoding is 
equivalent to solving the following problem: Find an assignment x E {0, 1}^ to variable nodes 
such that every path of length h in G w.r.t. vertex weights (— 1)"^" ■ A^/ degQ^v) has positive 
weight. By TheoremfTTl LP-decoding computes such a valid assignment x. The iterative NWMS 
decoding algorithm also computes such a valid assignment x provided that the local codes are 
restricted to SPC codes. 
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5.2 Punctured Tanner Codes 

Local-optimality characterization and its analysis remain valid under puncturing of a codeword. 
Puncturing of a code is specified by a subset of variable nodes that are not transmitted; this 
subset is called the puncturing pattern. A punctured code can be analyzed simply by zeroing 
the LLR values of the punctured variable nodes. 

How does puncturing affect the probability that a locally-optimal codeword exists? Con- 
sider a probabilistic analysis for local-optimality that is based on a simple union bound over 
the set of deviations of a Tanner code. We say that a variable node v participates in deviation (3 
if /3„ 7^ 0. Suppose that the puncturing pattern is chosen so that for every deviation (3, at most a 
constant fraction a of participating variable nodes are punctured. Then, the same union bound 
analysis can be applied directly to the punctured Tanner code. Such an analysis implies bounds 
that are similar to the bounds obtained for the (unpunctured) Tanner code; the only difference 
is that some parameters need to be scaled by a constant factor that is proportional to a. 

5.3 Analysis of Repeat- Accumulate Codes via Local-Optimality 

Feldman and Karger [FK021 IFK04n introduced the concept of linear-programming based de- 
coding for repeat-accumulate RA(g) turbo-like codes. In an RA(g) codes: (1) an information 
word is repeated q times, (2) the repeated information word is permuted by an interleaver, and 
(3) codeword bit i equals to the parity of the sum of the first i bits (prefix) of the permuted 
repeated information word. For RA(2) codes over the BSC, they proved that the word error 
probability of LP-decoding is bounded by an inverse polynomial in the code length, given a 
certain constant threshold on the noise. A similar claim was also proved for the BI-AWGN 
channel. These bounds were further improved based on exact combinatorial characterization 
of an error event and a refined algorithmic analysis [ HEOSI . 

Recently, Goldenberg and Burshtein [iGBllJ generalized the analysis of Feldman and Karger IIFK04n 
to RA(g) codes with even repetition g ^ 4. For this family of codes, they proved inverse poly- 
nomial bounds in the code length on the word error probability of LP-decoding. These bounds 
are based on analyzing complicated graph structures, called hyper-promenades in hypergraphs, 
that were defined in f FK04ll . 

In fact, the same bounds can be obtained by local-optimality with deviations induced by 
short paths. The idea is to consider (non-systematic) irregular repeat-accumulate codes with 
even repetition factors as punctured Tanner codes (as illustrated in Figured). Namely, set the 
LLR of each systematic variable node to zero. Notice that, for every backtrackless path in 
Tanner graphs of repeat accumulate codes, at most half of the variable nodes are systematic. 
Therefore, at most half of the variable nodes in each deviation of path-based local-optimality 
are punctured. We can now analyze the word error probability by replacing | by | in the 
proofs of Theorems [T2] and [T3l Hence the unified technique of local-optimality with deviations 
induced by short paths yields the same results proved by Feldman and Karger [IFK04II and 
Goldenberg and Burshtein [GBll]. Improving these bounds on successful decoding of RA(g) 
codes (for g > 3) remains an intriguing open question. 
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Codeword Variable Nodes 



Figure 1: Tanner graph representation of an irregular repeat- accumulate code. 
The variable nodes in the top row of nodes (illustrated by white circles) correspond to the k systematic 
bits. The repetition of every systematic bit corresponds to the degree qi of each systematic variable 
node (we assume that qi is even for every i). The interleaving process corresponds to the permutation of 
edges. The prefix accumulation corresponds to the alternating chain of single-parity check code nodes 
(illustrated by plus-squares) and codeword variable nodes (denoted by black circles) in the bottom. 



6 Conclusions 



We present a simple application of the proof technique in [IADS09II for bounds on word error 
probability with LP-decoding. The set of deviations used for defining local-optimality is in- 
duced by paths in the Tanner graph. We apply the proof technique to a family of codes, called 
even Tanner codes, that contains repeat accumulate codes with a even repetition factors, LDPC 
codes with even left degrees, and expander codes with even variable node degrees and even 
weighted local codes. Inverse polynomial error bounds are proved for these codes for the BSC 
and AWGN channel. 

Stronger error bounds have been obtained for LDPC codes flKVOei |ADS09[ IHEUH and 
Tanner codes [.EH 111 (without the restriction to even degrees and even weighted local codes) 
by considering more complicated graphical structures and a more sophisticated analysis. In 
these cases, inverse exponential error bounds and improved bounds on noise thresholds were 
presented for regular codes whose Tanner graphs have logarithmic girth. For example, the 
local-optimality characterization for Tanner codes presented in [ EHllll is based on projections 
of weighted subtrees in computation trees of the Tanner graph. The error bounds in this case are 
based on an analysis of a sum-min-sum random process on trees. While this local-optimality 
characterization applies to any regular and irregular Tanner code, the probabilistic analysis and 
the error bounds were restricted to regular Tanner codes. The simplicity of local-optimality 
based on paths enables us to obtain (weak) bounds even for irregular codes. Two interesting 
open questions related to proving stronger bounds on the error probability are (i) extend the 
analysis of inverse exponential bounds to irregular Tanner graphs, and (ii) obtain bounds with 
respect to local-optimality even "beyond the girth". 
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